<?xml version='1.0' encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <link href="cpip.css" rel="stylesheet" type="text/css" />
    <title>File: /Users/paulross/dev/linux/linux-3.13/include/linux/percpu-refcount.h</title>
  </head>
  <body>
    <h1>File: /Users/paulross/dev/linux/linux-3.13/include/linux/percpu-refcount.h</h1>
    <p>Green shading in the line number column
means the source is part of the translation unit, red means it is conditionally excluded.
Highlighted line numbers link to the translation unit page. Highlighted macros link to
the macro page.</p>
    <pre><a name="1" /><span class="True">       1:</span> <span class="k">/*</span>
<a name="2" /><span class="True">       2:</span> <span class="k"> * Percpu refcounts:</span>
<a name="3" /><span class="True">       3:</span> <span class="k"> * (C) 2012 Google, Inc.</span>
<a name="4" /><span class="True">       4:</span> <span class="k"> * Author: Kent Overstreet &lt;koverstreet@google.com&gt;</span>
<a name="5" /><span class="True">       5:</span> <span class="k"> *</span>
<a name="6" /><span class="True">       6:</span> <span class="k"> * This implements a refcount with similar semantics to atomic_t - atomic_inc(),</span>
<a name="7" /><span class="True">       7:</span> <span class="k"> * atomic_dec_and_test() - but percpu.</span>
<a name="8" /><span class="True">       8:</span> <span class="k"> *</span>
<a name="9" /><span class="True">       9:</span> <span class="k"> * There&apos;s one important difference between percpu refs and normal atomic_t</span>
<a name="10" /><span class="True">      10:</span> <span class="k"> * refcounts; you have to keep track of your initial refcount, and then when you</span>
<a name="11" /><span class="True">      11:</span> <span class="k"> * start shutting down you call percpu_ref_kill() _before_ dropping the initial</span>
<a name="12" /><span class="True">      12:</span> <span class="k"> * refcount.</span>
<a name="13" /><span class="True">      13:</span> <span class="k"> *</span>
<a name="14" /><span class="True">      14:</span> <span class="k"> * The refcount will have a range of 0 to ((1U &lt;&lt; 31) - 1), i.e. one bit less</span>
<a name="15" /><span class="True">      15:</span> <span class="k"> * than an atomic_t - this is because of the way shutdown works, see</span>
<a name="16" /><span class="True">      16:</span> <span class="k"> * percpu_ref_kill()/PCPU_COUNT_BIAS.</span>
<a name="17" /><span class="True">      17:</span> <span class="k"> *</span>
<a name="18" /><span class="True">      18:</span> <span class="k"> * Before you call percpu_ref_kill(), percpu_ref_put() does not check for the</span>
<a name="19" /><span class="True">      19:</span> <span class="k"> * refcount hitting 0 - it can&apos;t, if it was in percpu mode. percpu_ref_kill()</span>
<a name="20" /><span class="True">      20:</span> <span class="k"> * puts the ref back in single atomic_t mode, collecting the per cpu refs and</span>
<a name="21" /><span class="True">      21:</span> <span class="k"> * issuing the appropriate barriers, and then marks the ref as shutting down so</span>
<a name="22" /><span class="True">      22:</span> <span class="k"> * that percpu_ref_put() will check for the ref hitting 0.  After it returns,</span>
<a name="23" /><span class="True">      23:</span> <span class="k"> * it&apos;s safe to drop the initial ref.</span>
<a name="24" /><span class="True">      24:</span> <span class="k"> *</span>
<a name="25" /><span class="True">      25:</span> <span class="k"> * USAGE:</span>
<a name="26" /><span class="True">      26:</span> <span class="k"> *</span>
<a name="27" /><span class="True">      27:</span> <span class="k"> * See fs/aio.c for some example usage; it&apos;s used there for struct kioctx, which</span>
<a name="28" /><span class="True">      28:</span> <span class="k"> * is created when userspaces calls io_setup(), and destroyed when userspace</span>
<a name="29" /><span class="True">      29:</span> <span class="k"> * calls io_destroy() or the process exits.</span>
<a name="30" /><span class="True">      30:</span> <span class="k"> *</span>
<a name="31" /><span class="True">      31:</span> <span class="k"> * In the aio code, kill_ioctx() is called when we wish to destroy a kioctx; it</span>
<a name="32" /><span class="True">      32:</span> <span class="k"> * calls percpu_ref_kill(), then hlist_del_rcu() and sychronize_rcu() to remove</span>
<a name="33" /><span class="True">      33:</span> <span class="k"> * the kioctx from the proccess&apos;s list of kioctxs - after that, there can&apos;t be</span>
<a name="34" /><span class="True">      34:</span> <span class="k"> * any new users of the kioctx (from lookup_ioctx()) and it&apos;s then safe to drop</span>
<a name="35" /><span class="True">      35:</span> <span class="k"> * the initial ref with percpu_ref_put().</span>
<a name="36" /><span class="True">      36:</span> <span class="k"> *</span>
<a name="37" /><span class="True">      37:</span> <span class="k"> * Code that does a two stage shutdown like this often needs some kind of</span>
<a name="38" /><span class="True">      38:</span> <span class="k"> * explicit synchronization to ensure the initial refcount can only be dropped</span>
<a name="39" /><span class="True">      39:</span> <span class="k"> * once - percpu_ref_kill() does this for you, it returns true once and false if</span>
<a name="40" /><span class="True">      40:</span> <span class="k"> * someone else already called it. The aio code uses it this way, but it&apos;s not</span>
<a name="41" /><span class="True">      41:</span> <span class="k"> * necessary if the code has some other mechanism to synchronize teardown.</span>
<a name="42" /><span class="True">      42:</span> <span class="k"> * around.</span>
<a name="43" /><span class="True">      43:</span> <span class="k"> */</span>
<a name="44" /><span class="True">      44:</span> 
<a name="45" /><span class="True">      45:</span> <span class="f">#</span><span class="n">ifndef</span> <a href="cpu.c_macros_noref.html#_X0xJTlVYX1BFUkNQVV9SRUZDT1VOVF9IXzA_"><span class="b">_LINUX_PERCPU_REFCOUNT_H</span></a>
<a name="46" /><span class="True">      46:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_noref.html#_X0xJTlVYX1BFUkNQVV9SRUZDT1VOVF9IXzA_"><span class="b">_LINUX_PERCPU_REFCOUNT_H</span></a>
<a name="47" /><span class="True">      47:</span> 
<a name="48" /><span class="True">      48:</span> <span class="f">#</span><span class="n">include</span> <span class="f">&lt;</span><span class="b">linux</span><span class="f">/</span><span class="b">atomic</span><span class="f">.</span><span class="b">h</span><span class="f">&gt;</span>
<a name="49" /><span class="True">      49:</span> <span class="f">#</span><span class="n">include</span> <span class="f">&lt;</span><span class="b">linux</span><span class="f">/</span><span class="b">kernel</span><span class="f">.</span><span class="b">h</span><span class="f">&gt;</span>
<a name="50" /><span class="True">      50:</span> <span class="f">#</span><span class="n">include</span> <span class="f">&lt;</span><span class="b">linux</span><span class="f">/</span><span class="b">percpu</span><span class="f">.</span><span class="b">h</span><span class="f">&gt;</span>
<a name="51" /><span class="True">      51:</span> <span class="f">#</span><span class="n">include</span> <span class="f">&lt;</span><span class="b">linux</span><span class="f">/</span><span class="b">rcupdate</span><span class="f">.</span><span class="b">h</span><span class="f">&gt;</span>
<a name="52" /><span class="True">      52:</span> 
<a name="53" /><span class="True">      53:</span> <span class="m">struct</span> <span class="b">percpu_ref</span><span class="f">;</span>
<a name="54" /><span class="True">      54:</span> <span class="m">typedef</span> <span class="m">void</span> <span class="f">(</span><span class="b">percpu_ref_func_t</span><span class="f">)</span><span class="f">(</span><span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">*</span><span class="f">)</span><span class="f">;</span>
<a name="55" /><span class="True">      55:</span> 
<a name="56" /><span class="True">      56:</span> <span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">{</span>
<a name="57" /><span class="True">      57:</span>     <span class="b">atomic_t</span>        <span class="b">count</span><span class="f">;</span>
<a name="58" /><span class="True">      58:</span>     <span class="k">/*</span>
<a name="59" /><span class="True">      59:</span> <span class="k">     * The low bit of the pointer indicates whether the ref is in percpu</span>
<a name="60" /><span class="True">      60:</span> <span class="k">     * mode; if set, then get/put will manipulate the atomic_t (this is a</span>
<a name="61" /><span class="True">      61:</span> <span class="k">     * hack because we need to keep the pointer around for</span>
<a name="62" /><span class="True">      62:</span> <span class="k">     * percpu_ref_kill_rcu())</span>
<a name="63" /><span class="True">      63:</span> <span class="k">     */</span>
<a name="64" /><span class="True">      64:</span>     <span class="m">unsigned</span> <a href="cpu.c_macros_ref.html#_X19wZXJjcHVfMA__"><span class="b">__percpu</span></a>    <span class="f">*</span><span class="b">pcpu_count</span><span class="f">;</span>
<a name="65" /><span class="True">      65:</span>     <span class="b">percpu_ref_func_t</span>    <span class="f">*</span><span class="b">release</span><span class="f">;</span>
<a name="66" /><span class="True">      66:</span>     <span class="b">percpu_ref_func_t</span>    <span class="f">*</span><span class="b">confirm_kill</span><span class="f">;</span>
<a name="67" /><span class="True">      67:</span>     <span class="m">struct</span> <a href="cpu.c_macros_ref.html#_cmN1X2hlYWRfMA__"><span class="b">rcu_head</span></a>        <span class="b">rcu</span><span class="f">;</span>
<a name="68" /><span class="True">      68:</span> <span class="f">}</span><span class="f">;</span>
<a name="69" /><span class="True">      69:</span> 
<a name="70" /><span class="True">      70:</span> <span class="m">int</span> <a href="cpu.c_macros_ref.html#_X19tdXN0X2NoZWNrXzE_"><span class="b">__must_check</span></a> <span class="b">percpu_ref_init</span><span class="f">(</span><span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">*</span><span class="b">ref</span><span class="f">,</span>
<a name="71" /><span class="True">      71:</span>                  <span class="b">percpu_ref_func_t</span> <span class="f">*</span><span class="b">release</span><span class="f">)</span><span class="f">;</span>
<a name="72" /><span class="True">      72:</span> <span class="m">void</span> <span class="b">percpu_ref_cancel_init</span><span class="f">(</span><span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">*</span><span class="b">ref</span><span class="f">)</span><span class="f">;</span>
<a name="73" /><span class="True">      73:</span> <span class="m">void</span> <span class="b">percpu_ref_kill_and_confirm</span><span class="f">(</span><span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">*</span><span class="b">ref</span><span class="f">,</span>
<a name="74" /><span class="True">      74:</span>                  <span class="b">percpu_ref_func_t</span> <span class="f">*</span><span class="b">confirm_kill</span><span class="f">)</span><span class="f">;</span>
<a name="75" /><span class="True">      75:</span> 
<a name="76" /><span class="True">      76:</span> <span class="k">/**</span>
<a name="77" /><span class="True">      77:</span> <span class="k"> * percpu_ref_kill - drop the initial ref</span>
<a name="78" /><span class="True">      78:</span> <span class="k"> * @ref: percpu_ref to kill</span>
<a name="79" /><span class="True">      79:</span> <span class="k"> *</span>
<a name="80" /><span class="True">      80:</span> <span class="k"> * Must be used to drop the initial ref on a percpu refcount; must be called</span>
<a name="81" /><span class="True">      81:</span> <span class="k"> * precisely once before shutdown.</span>
<a name="82" /><span class="True">      82:</span> <span class="k"> *</span>
<a name="83" /><span class="True">      83:</span> <span class="k"> * Puts @ref in non percpu mode, then does a call_rcu() before gathering up the</span>
<a name="84" /><span class="True">      84:</span> <span class="k"> * percpu counters and dropping the initial ref.</span>
<a name="85" /><span class="True">      85:</span> <span class="k"> */</span>
<a name="86" /><span class="True">      86:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">percpu_ref_kill</span><span class="f">(</span><span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">*</span><span class="b">ref</span><span class="f">)</span>
<a name="87" /><span class="True">      87:</span> <span class="f">{</span>
<a name="88" /><span class="True">      88:</span>     <span class="m">return</span> <span class="b">percpu_ref_kill_and_confirm</span><span class="f">(</span><span class="b">ref</span><span class="f">,</span> <a href="cpu.c_macros_ref.html#_TlVMTF8w"><span class="b">NULL</span></a><span class="f">)</span><span class="f">;</span>
<a name="89" /><span class="True">      89:</span> <span class="f">}</span>
<a name="90" /><span class="True">      90:</span> 
<a name="91" /><span class="True">      91:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_ref.html#_UENQVV9TVEFUVVNfQklUU18w"><span class="b">PCPU_STATUS_BITS</span></a>    <span class="c">2</span>
<a name="92" /><span class="True">      92:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_ref.html#_UENQVV9TVEFUVVNfTUFTS18w"><span class="b">PCPU_STATUS_MASK</span></a>    <span class="f">(</span><span class="f">(</span><span class="c">1</span> <span class="f">&lt;&lt;</span> <a href="cpu.c_macros_ref.html#_UENQVV9TVEFUVVNfQklUU18w"><span class="b">PCPU_STATUS_BITS</span></a><span class="f">)</span> <span class="f">-</span> <span class="c">1</span><span class="f">)</span>
<a name="93" /><span class="True">      93:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_ref.html#_UENQVV9SRUZfUFRSXzA_"><span class="b">PCPU_REF_PTR</span></a>        <span class="c">0</span>
<a name="94" /><span class="True">      94:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_noref.html#_UENQVV9SRUZfREVBRF8w"><span class="b">PCPU_REF_DEAD</span></a>        <span class="c">1</span>
<a name="95" /><span class="True">      95:</span> 
<a name="96" /><span class="True">      96:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_ref.html#_UkVGX1NUQVRVU18w"><span class="b">REF_STATUS</span></a><span class="f">(</span><span class="b">count</span><span class="f">)</span>    <span class="f">(</span><span class="f">(</span><span class="f">(</span><span class="m">unsigned</span> <span class="m">long</span><span class="f">)</span> <span class="b">count</span><span class="f">)</span> <span class="f">&amp;</span> <a href="cpu.c_macros_ref.html#_UENQVV9TVEFUVVNfTUFTS18w"><span class="b">PCPU_STATUS_MASK</span></a><span class="f">)</span>
<a name="97" /><span class="True">      97:</span> 
<a name="98" /><span class="True">      98:</span> <span class="k">/**</span>
<a name="99" /><span class="True">      99:</span> <span class="k"> * percpu_ref_get - increment a percpu refcount</span>
<a name="100" /><span class="True">     100:</span> <span class="k"> * @ref: percpu_ref to get</span>
<a name="101" /><span class="True">     101:</span> <span class="k"> *</span>
<a name="102" /><span class="True">     102:</span> <span class="k"> * Analagous to atomic_inc().</span>
<a name="103" /><span class="True">     103:</span> <span class="k">  */</span>
<a name="104" /><span class="True">     104:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">percpu_ref_get</span><span class="f">(</span><span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">*</span><span class="b">ref</span><span class="f">)</span>
<a name="105" /><span class="True">     105:</span> <span class="f">{</span>
<a name="106" /><span class="True">     106:</span>     <span class="m">unsigned</span> <a href="cpu.c_macros_ref.html#_X19wZXJjcHVfMA__"><span class="b">__percpu</span></a> <span class="f">*</span><span class="b">pcpu_count</span><span class="f">;</span>
<a name="107" /><span class="True">     107:</span> 
<a name="108" /><span class="True">     108:</span>     <span class="b">rcu_read_lock_sched</span><span class="f">(</span><span class="f">)</span><span class="f">;</span>
<a name="109" /><span class="True">     109:</span> 
<a name="110" /><span class="True">     110:</span>     <span class="b">pcpu_count</span> <span class="f">=</span> <a href="cpu.c_macros_ref.html#_QUNDRVNTX09OQ0VfMA__"><span class="b">ACCESS_ONCE</span></a><span class="f">(</span><span class="b">ref</span><span class="f">-&gt;</span><span class="b">pcpu_count</span><span class="f">)</span><span class="f">;</span>
<a name="111" /><span class="True">     111:</span> 
<a name="112" /><span class="True">     112:</span>     <span class="m">if</span> <span class="f">(</span><a href="cpu.c_macros_ref.html#_bGlrZWx5XzA_"><span class="b">likely</span></a><span class="f">(</span><a href="cpu.c_macros_ref.html#_UkVGX1NUQVRVU18w"><span class="b">REF_STATUS</span></a><span class="f">(</span><span class="b">pcpu_count</span><span class="f">)</span> <span class="f">==</span> <a href="cpu.c_macros_ref.html#_UENQVV9SRUZfUFRSXzA_"><span class="b">PCPU_REF_PTR</span></a><span class="f">)</span><span class="f">)</span>
<a name="113" /><span class="True">     113:</span>         <a href="cpu.c_macros_ref.html#_X190aGlzX2NwdV9pbmNfMA__"><span class="b">__this_cpu_inc</span></a><span class="f">(</span><span class="f">*</span><span class="b">pcpu_count</span><span class="f">)</span><span class="f">;</span>
<a name="114" /><span class="True">     114:</span>     <span class="m">else</span>
<a name="115" /><span class="True">     115:</span>         <span class="b">atomic_inc</span><span class="f">(</span><span class="f">&amp;</span><span class="b">ref</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span><span class="f">;</span>
<a name="116" /><span class="True">     116:</span> 
<a name="117" /><span class="True">     117:</span>     <span class="b">rcu_read_unlock_sched</span><span class="f">(</span><span class="f">)</span><span class="f">;</span>
<a name="118" /><span class="True">     118:</span> <span class="f">}</span>
<a name="119" /><span class="True">     119:</span> 
<a name="120" /><span class="True">     120:</span> <span class="k">/**</span>
<a name="121" /><span class="True">     121:</span> <span class="k"> * percpu_ref_tryget - try to increment a percpu refcount</span>
<a name="122" /><span class="True">     122:</span> <span class="k"> * @ref: percpu_ref to try-get</span>
<a name="123" /><span class="True">     123:</span> <span class="k"> *</span>
<a name="124" /><span class="True">     124:</span> <span class="k"> * Increment a percpu refcount unless it has already been killed.  Returns</span>
<a name="125" /><span class="True">     125:</span> <span class="k"> * %true on success; %false on failure.</span>
<a name="126" /><span class="True">     126:</span> <span class="k"> *</span>
<a name="127" /><span class="True">     127:</span> <span class="k"> * Completion of percpu_ref_kill() in itself doesn&apos;t guarantee that tryget</span>
<a name="128" /><span class="True">     128:</span> <span class="k"> * will fail.  For such guarantee, percpu_ref_kill_and_confirm() should be</span>
<a name="129" /><span class="True">     129:</span> <span class="k"> * used.  After the confirm_kill callback is invoked, it&apos;s guaranteed that</span>
<a name="130" /><span class="True">     130:</span> <span class="k"> * no new reference will be given out by percpu_ref_tryget().</span>
<a name="131" /><span class="True">     131:</span> <span class="k"> */</span>
<a name="132" /><span class="True">     132:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">bool</span> <span class="b">percpu_ref_tryget</span><span class="f">(</span><span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">*</span><span class="b">ref</span><span class="f">)</span>
<a name="133" /><span class="True">     133:</span> <span class="f">{</span>
<a name="134" /><span class="True">     134:</span>     <span class="m">unsigned</span> <a href="cpu.c_macros_ref.html#_X19wZXJjcHVfMA__"><span class="b">__percpu</span></a> <span class="f">*</span><span class="b">pcpu_count</span><span class="f">;</span>
<a name="135" /><span class="True">     135:</span>     <span class="m">int</span> <span class="b">ret</span> <span class="f">=</span> <span class="m">false</span><span class="f">;</span>
<a name="136" /><span class="True">     136:</span> 
<a name="137" /><span class="True">     137:</span>     <span class="b">rcu_read_lock_sched</span><span class="f">(</span><span class="f">)</span><span class="f">;</span>
<a name="138" /><span class="True">     138:</span> 
<a name="139" /><span class="True">     139:</span>     <span class="b">pcpu_count</span> <span class="f">=</span> <a href="cpu.c_macros_ref.html#_QUNDRVNTX09OQ0VfMA__"><span class="b">ACCESS_ONCE</span></a><span class="f">(</span><span class="b">ref</span><span class="f">-&gt;</span><span class="b">pcpu_count</span><span class="f">)</span><span class="f">;</span>
<a name="140" /><span class="True">     140:</span> 
<a name="141" /><span class="True">     141:</span>     <span class="m">if</span> <span class="f">(</span><a href="cpu.c_macros_ref.html#_bGlrZWx5XzA_"><span class="b">likely</span></a><span class="f">(</span><a href="cpu.c_macros_ref.html#_UkVGX1NUQVRVU18w"><span class="b">REF_STATUS</span></a><span class="f">(</span><span class="b">pcpu_count</span><span class="f">)</span> <span class="f">==</span> <a href="cpu.c_macros_ref.html#_UENQVV9SRUZfUFRSXzA_"><span class="b">PCPU_REF_PTR</span></a><span class="f">)</span><span class="f">)</span> <span class="f">{</span>
<a name="142" /><span class="True">     142:</span>         <a href="cpu.c_macros_ref.html#_X190aGlzX2NwdV9pbmNfMA__"><span class="b">__this_cpu_inc</span></a><span class="f">(</span><span class="f">*</span><span class="b">pcpu_count</span><span class="f">)</span><span class="f">;</span>
<a name="143" /><span class="True">     143:</span>         <span class="b">ret</span> <span class="f">=</span> <span class="m">true</span><span class="f">;</span>
<a name="144" /><span class="True">     144:</span>     <span class="f">}</span>
<a name="145" /><span class="True">     145:</span> 
<a name="146" /><span class="True">     146:</span>     <span class="b">rcu_read_unlock_sched</span><span class="f">(</span><span class="f">)</span><span class="f">;</span>
<a name="147" /><span class="True">     147:</span> 
<a name="148" /><span class="True">     148:</span>     <span class="m">return</span> <span class="b">ret</span><span class="f">;</span>
<a name="149" /><span class="True">     149:</span> <span class="f">}</span>
<a name="150" /><span class="True">     150:</span> 
<a name="151" /><span class="True">     151:</span> <span class="k">/**</span>
<a name="152" /><span class="True">     152:</span> <span class="k"> * percpu_ref_put - decrement a percpu refcount</span>
<a name="153" /><span class="True">     153:</span> <span class="k"> * @ref: percpu_ref to put</span>
<a name="154" /><span class="True">     154:</span> <span class="k"> *</span>
<a name="155" /><span class="True">     155:</span> <span class="k"> * Decrement the refcount, and if 0, call the release function (which was passed</span>
<a name="156" /><span class="True">     156:</span> <span class="k"> * to percpu_ref_init())</span>
<a name="157" /><span class="True">     157:</span> <span class="k"> */</span>
<a name="158" /><span class="True">     158:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">percpu_ref_put</span><span class="f">(</span><span class="m">struct</span> <span class="b">percpu_ref</span> <span class="f">*</span><span class="b">ref</span><span class="f">)</span>
<a name="159" /><span class="True">     159:</span> <span class="f">{</span>
<a name="160" /><span class="True">     160:</span>     <span class="m">unsigned</span> <a href="cpu.c_macros_ref.html#_X19wZXJjcHVfMA__"><span class="b">__percpu</span></a> <span class="f">*</span><span class="b">pcpu_count</span><span class="f">;</span>
<a name="161" /><span class="True">     161:</span> 
<a name="162" /><span class="True">     162:</span>     <span class="b">rcu_read_lock_sched</span><span class="f">(</span><span class="f">)</span><span class="f">;</span>
<a name="163" /><span class="True">     163:</span> 
<a name="164" /><span class="True">     164:</span>     <span class="b">pcpu_count</span> <span class="f">=</span> <a href="cpu.c_macros_ref.html#_QUNDRVNTX09OQ0VfMA__"><span class="b">ACCESS_ONCE</span></a><span class="f">(</span><span class="b">ref</span><span class="f">-&gt;</span><span class="b">pcpu_count</span><span class="f">)</span><span class="f">;</span>
<a name="165" /><span class="True">     165:</span> 
<a name="166" /><span class="True">     166:</span>     <span class="m">if</span> <span class="f">(</span><a href="cpu.c_macros_ref.html#_bGlrZWx5XzA_"><span class="b">likely</span></a><span class="f">(</span><a href="cpu.c_macros_ref.html#_UkVGX1NUQVRVU18w"><span class="b">REF_STATUS</span></a><span class="f">(</span><span class="b">pcpu_count</span><span class="f">)</span> <span class="f">==</span> <a href="cpu.c_macros_ref.html#_UENQVV9SRUZfUFRSXzA_"><span class="b">PCPU_REF_PTR</span></a><span class="f">)</span><span class="f">)</span>
<a name="167" /><span class="True">     167:</span>         <a href="cpu.c_macros_ref.html#_X190aGlzX2NwdV9kZWNfMA__"><span class="b">__this_cpu_dec</span></a><span class="f">(</span><span class="f">*</span><span class="b">pcpu_count</span><span class="f">)</span><span class="f">;</span>
<a name="168" /><span class="True">     168:</span>     <span class="m">else</span> <span class="m">if</span> <span class="f">(</span><a href="cpu.c_macros_ref.html#_dW5saWtlbHlfMA__"><span class="b">unlikely</span></a><span class="f">(</span><span class="b">atomic_dec_and_test</span><span class="f">(</span><span class="f">&amp;</span><span class="b">ref</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span><span class="f">)</span><span class="f">)</span>
<a name="169" /><span class="True">     169:</span>         <span class="b">ref</span><span class="f">-&gt;</span><span class="b">release</span><span class="f">(</span><span class="b">ref</span><span class="f">)</span><span class="f">;</span>
<a name="170" /><span class="True">     170:</span> 
<a name="171" /><span class="True">     171:</span>     <span class="b">rcu_read_unlock_sched</span><span class="f">(</span><span class="f">)</span><span class="f">;</span>
<a name="172" /><span class="True">     172:</span> <span class="f">}</span>
<a name="173" /><span class="True">     173:</span> 
<a name="174" /><span class="True">     174:</span> <span class="f">#</span><span class="n">endif</span>
<a name="175" /><span class="True">     175:</span> </pre>
  </body>
</html>
